We explore unifying a neural segmenter with two-pass cascaded encoder ASR into a single model. A key challenge is allowing the segmenter (which runs in real-time, synchronously with the decoder) to finalize the 2nd pass (which runs 900 ms behind real-time) without introducing user-perceived latency or deletion errors during inference. We propose a design where the neural segmenter is integrated with the causal 1st pass decoder to emit a end-of-segment (EOS) signal in real-time. The EOS signal is then used to finalize the non-causal 2nd pass. We experiment with different ways to finalize the 2nd pass, and find that a novel dummy frame injection strategy allows for simultaneous high quality 2nd pass results and low finalization latency. On a real-world long-form captioning task (YouTube), we achieve 2.4% relative WER and 140 ms EOS latency gains over a baseline VAD-based segmenter with the same cascaded encoder.

语言识别对于自动语音识别（ASR）中的许多下游任务至关重要，并且有益于将多语言端到端的ASR集成为附加任务。在本文中，我们建议通过集成每帧语言标识符（LID）预测器来修改基于层压编码器的复发神经网络传感器（RNN-T）模型的结构。带有级联编码器的RNN-T可以使用不右键的第一通用解码来实现较低延迟的流动ASR，并使用二频道解码使用更长的右文本实现较低的单词错误率（WERS）。通过利用当前文章中的这种差异和统计池的流传输实现，该建议的方法可以实现准确的流盖预测，而几乎没有额外的测试时间成本。语音搜索数据集的实验结果具有9个语言语言位置，表明所提出的方法平均达到96.2％的盖子预测准确性，而与输入中的Oracle盖相同的二次通用方法。

设备的端到端（E2E）模型已显示出对质量和延迟的英语语音搜索任务的常规模型的改进。 E2E模型还显示了多语言自动语音识别（ASR）的有希望的结果。在本文中，我们将以前的容量解决方案扩展到流应用程序，并提出流媒体多语言E2E ASR系统，该系统在设备上完全运行，质量和延迟与单个单语言模型相当。为了实现这一目标，我们提出了一个编码器端量模型和一个终端（EOU）联合层，以提高质量和延迟权衡。我们的系统以语言不可知论的方式构建，允许它实时支持本条件的代码切换。为了解决大型模型的可行性问题，我们进行了设备分析，并用最近开发的嵌入解码器代替了耗时的LSTM解码器。通过这些更改，我们设法在不到实时的时间内在移动设备上运行了这样的系统。

在启用语音的应用程序中，一个预定的热词在同时用来激活设备以便进行查询。 toavoid重复一个热词，我们提出了一个端到端的流（E2E）打算查询检测器，该查询检测器识别向设备指向的发音，并滤除针对设备的其他发出内容。提出的方法将预期的查询检测器置于E2E模型中，该模型将语音识别的不同组件折叠成一个神经网络。E2E对台面解码和预期的查询检测进行建模，也使我们可以基于早期的部分偏置检测结果， ，这对于减少潜伏期和使系统响应很重要。我们证明，与独立的预期检测器相比，检测准确性和600个MSLATENCE的相对相对改善的相对提高一级误差率（EER）的相对提高了22％。在我们的实验中，提出的模型检测用户正在用用户开始讲话后，用8.7％的Eerwithin与设备进行对话。

尽管流媒体助手系统已在许多应用中使用，但该系统通常集中于不自然的单次交互，假设来自单个语音查询的输入毫不犹豫地或不足。但是，除了反弹之外，常见的对话说法通常涉及多个转弯的查询。这些疏远包括暂停思考，犹豫，延长单词，填补的停顿和重复的短语。这使得通过对话演讲进行语音识别，其中包括有多个查询，这是一项具有挑战性的任务。为了更好地建模对话互动，至关重要的是，歧视汇率和查询的结束至关重要，以使用户能够在用户完成时，同时使系统尽快做出响应，以使用户保持地板的折衷。在本文中，我们提出了一个基于端到端（E2E）语音识别器的转折预测指标。我们的最佳系统是通过共同优化ASR任务并检测用户何时停止思考或完成口语来获得的。所提出的方法显示，在预测真正的转弯率的97％以上的召回率和85％的精度率中，在设计集中仅100毫秒延迟，设计了4种类型的对话说法中插入4种散布。

在长时间到数小时的长时间话语中，提高端到端ASR模型的性能是语音识别的持续挑战。一个常见的解决方案是使用单独的语音活动检测器（VAD）事先将音频分割，该声音活动检测器（VAD）纯粹基于声音/非语音信息来决定段边界位置。但是，VAD细分器可能是现实世界语音的最佳选择，例如，一个完整的句子应该整体上可能包含犹豫（“设置... 5点钟的警报”） 。我们建议用端到端的ASR模型替换VAD，能够以流方式预测段边界，从而使细分决定不仅在更好的声学特征上，而且还可以在解码文本的语义特征上进行，并具有可忽略的额外功能计算。在现实世界长音频（YouTube）的实验中，长度长达30分钟，我们证明了相对改善的8.5％，并且与VAD段基线相比，中位段延迟潜伏期的中位数延迟延迟减少了250毫秒。 - ART构象体RNN-T模型。

The recent increase in public and academic interest in preserving biodiversity has led to the growth of the field of conservation technology. This field involves designing and constructing tools that utilize technology to aid in the conservation of wildlife. In this article, we will use case studies to demonstrate the importance of designing conservation tools with human-wildlife interaction in mind and provide a framework for creating successful tools. These case studies include a range of complexities, from simple cat collars to machine learning and game theory methodologies. Our goal is to introduce and inform current and future researchers in the field of conservation technology and provide references for educating the next generation of conservation technologists. Conservation technology not only has the potential to benefit biodiversity but also has broader impacts on fields such as sustainability and environmental protection. By using innovative technologies to address conservation challenges, we can find more effective and efficient solutions to protect and preserve our planet's resources.

We present Muse, a text-to-image Transformer model that achieves state-of-the-art image generation performance while being significantly more efficient than diffusion or autoregressive models. Muse is trained on a masked modeling task in discrete token space: given the text embedding extracted from a pre-trained large language model (LLM), Muse is trained to predict randomly masked image tokens. Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding. The use of a pre-trained LLM enables fine-grained language understanding, translating to high-fidelity image generation and the understanding of visual concepts such as objects, their spatial relationships, pose, cardinality etc. Our 900M parameter model achieves a new SOTA on CC3M, with an FID score of 6.06. The Muse 3B parameter model achieves an FID of 7.88 on zero-shot COCO evaluation, along with a CLIP score of 0.32. Muse also directly enables a number of image editing applications without the need to fine-tune or invert the model: inpainting, outpainting, and mask-free editing. More results are available at https://muse-model.github.io

An unbiased scene graph generation (SGG) algorithm referred to as Skew Class-balanced Re-weighting (SCR) is proposed for considering the unbiased predicate prediction caused by the long-tailed distribution. The prior works focus mainly on alleviating the deteriorating performances of the minority predicate predictions, showing drastic dropping recall scores, i.e., losing the majority predicate performances. It has not yet correctly analyzed the trade-off between majority and minority predicate performances in the limited SGG datasets. In this paper, to alleviate the issue, the Skew Class-balanced Re-weighting (SCR) loss function is considered for the unbiased SGG models. Leveraged by the skewness of biased predicate predictions, the SCR estimates the target predicate weight coefficient and then re-weights more to the biased predicates for better trading-off between the majority predicates and the minority ones. Extensive experiments conducted on the standard Visual Genome dataset and Open Image V4 \& V6 show the performances and generality of the SCR with the traditional SGG models.

With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few training examples. It has been a new trend exploring ICL to evaluate and extrapolate the ability of LLMs. In this paper, we aim to survey and summarize the progress, challenges, and future work in ICL. We first present a formal definition of ICL and clarify its correlation to related studies. Then, we organize and discuss advanced techniques of ICL, including training strategies, prompting strategies, and so on. Finally, we present the challenges of ICL and provide potential directions for further research. We hope our work can encourage more research on uncovering how ICL works and improving ICL in future work.